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ABSTRACT 

This prototype instructional material 
source of information about curriculum and program de 
materials^ instructional and training models^ and oth 
use in educational research^ deveiopmentr diffusion^ 
(RDD6E) training. Section I describes products which^ 
partr have been developed with support from the Natio 
Education's Task Force on Researcher Training. Some o 
are now or soon will be available for general use in 
nondegree training programs. Others are still in deve 
be made available for field testing or reviewed for i 
purposes. The more than 30 other products in the cata 
into four sections: Training Products for Other Educa 
Professionals, Training Models and Techniques, Traini 
and Supplementary Materials. The title and the develo 
each product and a brief abstract are provided for th 
these sections. (Author) 
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PREFACE 



This/ prototype Instructional Materials Catalog is a source of in- 
fovmtion about curriculum and program developments, materials, 
instructional and training models, and other resources for use in 
educational research, development, diffusion, and evaluation (RDD&E) 
traimng. 

Section I describes products which, for tl|ie most part, have been 
developed with support from the National Institute of Education's 
Task Force on Researcher Training. Some of these products are now 
or soon will be available for general usel in degree and non-degree 
training programs. Others are still in development, but may be 
made available for field testing or reviewed for information pur- 
poses. The content for each of the product descriptions was either 
submitted by the developer or extracted from reporting forms which 
the developer prepared for NIE. J 

The more than 30 other products in the Catalog are grouped into four 
sections: Training Products for Other Education Professionals, 
Training Models and Techniques, Training Materials, and Supplementary 
Materials. The title and the developer or source of each product 
and a brief abstract are provided for the products in these sections. 

The reader should note that this Catalog does not, by any means, 
include all relevant and available products. It represents a first 
attempt to con[f)ile, classify, organize, and annotate a small collec- 
tion of educational RDD&E training materials acquired thus far for 
an experimental project on RDD&E Information System Design being 
conducted by the Far West Laboratory. The Catalog will be updated 
and expanded, if the need for this kind of information is suffi- 
ciently evidenced. 
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I. TRAINING PRODUCTS FOR 
RESEARCH. DEVELOPMENT, DIFFUSION, AND EVALUATION 

PERSONNEL 



2 



I APPRAISING EDUCATIONAL RESEARCH: A CASE STUDY APPROACH 



This workbook is intended for use in improving the ability of learners 
to appraise critically educational research. The print materials consist of 
the following: an introductory statement about the nature of criticism, 
a statement about the contents of the materials and suggestions for use, 
and nine case studies. The problem areas in the nine cases cover: 
grading and student attitudes, evaluation of a Head Start program, 
prediction of long-term success, reinforcement and behavioral modifi- 
cation, effect of questioning procedures on student achievement, 
development of a concept of justice, factors affecting the validity 
of creativity assessments, experimentation with live animals in 
elementary school, and small group composition and productivity. The 
types of research in the cases include: status, prediction, experimental, 
case study, and philosophical analysis. Students read each article and 
special notes and then respond to orienting questions either verbally 
(if in a class situation) or in writing. They compare their responses 
to model appraisals. Comprehension of the articles requires neither 
statistical sophistication nor expertise in the substantive field. 

Research and research-related personnel, 
including students in research methods 
courses. 

In conjunction with a research methods 
course or separately. 



Workbook 



None specified 
Spring 1973 

Approximately $5 per workbook. 

Jason Millman and D. Bob Gowin, Department 
of Education, Stone Hall, Cornell University, 
Ithaca, New York 14850 

Prentice-Hall, Inc., Englewood Cliffs, 
New Jersey 



ERIC 



SuggeJiteJ U^e; 

MatzAycatb and 
EqiUpmnt Weeded: 

Tn6tAuctionaJi 
RzquAJKmzYitJb 

AvcUZabitiXy : 



Un,ban EduaitLon 



URBAN EDUCATIONAL RESEARCHERS TRAINING INSTITUTE 



The Institute provides an instructional model for use by school systems, 
community agencies, college and universities interested in training 
research and evaluation personnel Involved in urban school settings. 
Two six-credit courses were designed for the institute, sponsored by 
Howard University. They cover research proposal writing, statement of 
the problem, use of theory in the development of hypothesis, review of 
literature, research design-population selection, instrumentation, data 
collection and analyses, and report writing. Each course was a semester 
in length. Readings came from various research books supplied to the 
trainees. The trainees spent two weeks in classes developing a research 
proposal related to a problem in their home school systems. They are 
receiving monthly on-the-job supervision in carrying out their proposal, 
followed by two on-campus weekends for progress reporting and individual 
instruction. At the end of the project, they will spend two weeks on 
campus analyzing the data from their research project, writing up the 
findings, and receiving further training in research evaluation. 



McUeAA.al6 and 

AvcuLtab^ltLty : 
C06t6: 



Research and evaluation personnel in urban 
educational settings. 

As an instructional model for use by school 
systems, higher educational institutions, and 
staffs of R4D centers. 



10 modules 

Not specified 
October 1973 
Not specified 

Harriette and John McAdoo, School of Social Work, 
Howard University, Washington, D. C. 20001 
(202) 636-7300 

Not specified 
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B AN INTERVIEWING TRAINING MODULE 



This instructional package is designed for general use in training 
individuals or groups in the basic techniques of conducting survey 
research interviews. It can also be used to acquaint students with 
the interview as a data collection device~e-g-, in a research methods 
course--without unnecessarily concentrating on the practice of specific 
skills. The module package consists of the following components: 
three stop-action films; a set of structured role-playing activities 
to be used during the breaks in the films; two copies of professionally 
developed interview schedules which students administer outside of the 
training sessions; a set of question-by-question specifications for 
the schedules to assist in their usage; and a student manual which 
explains the basic skills involved in conducting an interview. Each 
component is designed to supplement the others, providing students 
with the skills they need, an opportunity to practice these skills in 
a relatively controlled setting, and an opportunity to practice them 
"in the field." Student evaluation materials are supplied with an 
instructor's manual. The entire package is nearly self-instructional, 
requiring little preparation on the part of the instructor- Although 
the module is designed to be used in three, one-hour training sessions, 
plus out-of-class assignments, this format may be altered at the instructor's 
option. 



ToAQQJt Audience.* 



Persons in college or university, educational or 
public health agency, or private corporation who 
need training in interviewing techniques. 



General, requires three classroom hours. 



tA(xt2Aicil6 and 



Three 16mm stop-action color films, 16mm film 
projector, simulation booklets, two interview 
schedules and specifications. Survey Research 
Interviewing Manual for the student, and 



procedural guide for instructor. 



JMtAuctLdnal 



Instructor helpful, but not required. 



Avallabtti^ty: 



1973 



Coi>-t6: 



Unknown 



Emil J. Haller, Associate Professor, Department 
of Education, 103 Stone Hall, Cornell University, 
Ithaca, New York 14850 (607) 256-2267 



None at present. 



TQA>t VzvzZopmzYtt/ 



■ PLANNING A TEST-A FILMSTRIP UNIT ON BASIC MEASUREMENT PRINCIPLES 



A filmstrip with associated audio track has been developed to cover the 
major planning steps in the development of a measurement instrument such 
as a test or questionnaire. The filmstrip addresses the following six 
questions: why am I testing, wi^at should I test, whom am I testing, 
what kinds of questions should I use, how long should my test be, and 
how difficult should my test be? A set of supplementary materials cover 
the following issues: learning how to develop tests, obtaining informa- 
tion about tests, preparing a test plan, kinds of test questions- 
advantages and disadvantages, reliability, and criterion-referenced 
tests. The supplementary materials provide an expanded treatment of 
some of the issues raised in the filmstrip, but their primary function 
is as a guide to appropriate literature for those individuals seeking 
an intensive treatment of topics related to their particular interest 
and needs. No student testing or evaluation materials are included in 
the package. No previous training in measurement is assumed. 



ToAgzt Audicncz: Persons engaged in or receiving training in 

evaluation, such as students in introductory 
educational psychology, measurement, or research 
training, or teachers, administrators, or other 
educational personnel. 

SuggeAt^d U4e: In graduate or undergraduate level courses or 

for inservice training courses or workshops. 



HatQJiLaJU and 

EqtUpnp^nt Hzzdzdt Filmstrip and audiotrack, filmstrip projector, 

printed supplementary materials (24pp). 



l¥U>t/UXCJXOYWUi 

RdqtuAomnt^ : Not specified 

AvcUlabAjUXy: 1973 
Co6t6: Not specified 

VzvzlopzK: John J. Fremer, Jr., Educational Testing Service, 

Princeton, New Jersey 08540 (609) 921-9000 



Educational Testing Service, Princeton, 
New Jersey 08540 



Re^eruch arid 
EvahicuUon 



B FEHR-PRACTICUM 

FEHR (which stands for "formative evaluation and heuristic research'*) 
-PRACTICUM is a computerized game which simulates the experience of a 
research and evaluation assistantship or practicum. The game provides 
practice in: selecting appropriate criterion measures; identifying 
relevant independent variables from a wider variety of available 
variables (e.g., ethnic groups, school, SES level, intelligence, age, 
curriculum content, etc.); designing a study to provide valid answers 
to an educational question; choosing research subjects via a sampling 
technique appropriate to the design; analyzing and interpreting data; 
and assessing the adequacy of an educational decision derived from an 
"experiment.*' The game is usually played by two or more teams, each 
consisting of three of four members. Each team "solves" a research/ 
evaluation problem. The teams must collect information about past 
research in the area and about the behavior of the research subjects 
(students and teachers). The records of previous research are housed 
in an Information Bank, a kind of simulated library, and both the 
research environment and the behavior of the research subjects are 
simulated by a computer program. 

ToAgeX Aadcence: Students in research and evaluation. 



SuggeAtzd U6e: 



Self-instructional simulation for a course of 
3-5 semester hours (requires three to four 
three-hour sessions). Instructor must supply 
statistical training needed. 



M(it2AMil6 and 
EquUprnzrvt Hzzdzd: 



Computer-based 



IvUitnjjLCJtionaZ 
RzqiiiAmmtii : 



Self-instructional 



AvcULcibltUtj: 



Not specified 



Undetermined 



LeVerne S. Collet, University of Michigan, 
421 5A School of Education, Ann Arbor, Michigan 
48104 (313) 764-8435 



Not specified 
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TRAINING IN 
PROCEDURES 



Vzvzlopmznt 

A STRATEGY FOR DEVELOPING EDUCATIONAL PRODUCTS AND 



The program is designed to train educators in the use of a strategy, the 
IDEALS concept, specifically developed for design or development work. 
The IDEALS (Ideal Design of Effective and Logical Systems) concept has 
four parts: (a) a prescriptive, universal framework or system for 
specifying any solution in terms of eight elements in five dimensions; 
(b) a design strategy of ten steps, dealing with functions and ideal 
system targets; (c) a focus on functions and targets to unite organiza- 
tional personnel in a positive participatory framework for continuing 
system design and improvement program; and (d) use of models only when 
helpful and appropriate. The sequence of activities begins with a three- 
week training course on the IDEALS concept design strategy and related 
concepts, followed by two months of application of the strategy to 
projects in the participants* organizations, a two-day review and 
question answering workshop for solving problems which developed during 
the application period* an'- <s final three-day report and wrap-up work- 
shop for evaluation. 



Ta/iQzt Audi met: 



MatoAiats and 
^qtUpfmnt Weeded: 



Educational professionals in critical decision- 
making positions at various organizational levels, 

Over a six-month period, beginning with three- 
week training course and interspersed with 
application experience and two workshops. 



Basic textbook. Work Design: A Systems Concept , 
plus transparencies, slides, overhead projector. 



Inbt/iuctionaZ 
ReqtuAe/nent4 : 



Availability: 
Vl^t^buto^: 



Instructor, with background in system design, 
needed for three-week course. 

Textbook to be published in September 1973. Other 
specifics for training still in field testing. 

Not yet available 

Gerald Nadler, Department of Industrial Engineering, 
University of Wisconsin, Madison, Wisconsin 53706 

Not selected 
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V^v^lopm^nt/V^6eJrl^ation/ 
Bva^uLotlon 

m A TRAINING PROGRAM IN EDUCATIONAL DEVELOPMENT, DISSEMINATION, AND 
■■ EVALUATION 

This comprehensive, transportable program 1s designed to train personnel 
In educational development, dissemination, and evaluation (DD&E) skills. 
The content of the program covers nine competence areas: planning and 
design. Information/data collection and organization, communication skills, 
developmental engineering, evaluation, analysis and definition, dissemina- 
tion and marketing, management, and dissemination of educational information. 
Training materials for each area are labelled a ''series." Each series 
consists of from four to seven "modules" or self-instructional units 
focusing on sets of related skills. Competence assessment batteries, con- 
sisting of Instruments such as trainee and supervisor ratings* job knowledge 
tests, simulation exercises, and product rating scales, are provided for 
each of the competence areas. The program features an instructional pattern 
called a triad, which consists of the student, an Instructional Resource 
Manager (IRM), and a work supervisor. The student works with the instruc- 
tional modules, performs activities and exercises, and, at the same time, 
applies skills in an actual work situation. The IRM functions as a guide 
and resource person, assigning modules and reviewing student progress on 
a work plan and on achievement of assigned modules. The work supervisor 
oversees the student's job activities, provides on-the-job Instruction 
as needed, and provides feedback and evaluation to the IRM. 



Tcvigzt Aacttewce: 
Suggest ttd U6^: 



Personnel planning to enter or already in DD&E 
positions. 

Three levels: (1) a comprehensive, degree- 
oriented program coordinated at a college or 
university; (2) use of series or clusters of 
modules as required by the learner; or (3) use 
of individual modules as required. 



Mcut2JUcuU and 
Equiprmnt Hzzdzdi 



About 30 printed modules, competence assessment 
Instruments for each module, catalogs of instruc- 
tional resources and assessment materials, and 
guides for quality control and implementation 
of program. 



iMViucttoncit 



AvaiZabiZitij: 



For a comprehensive use, instructional, college 
administrative staff, and R&D agency staff support 
needed. Modules are self-administered, requiring 
only supportive instructional staff. 

For field test use only. Fall 1973. 

None at present. 

The Far West Consortium, Far West Laboratory for 
Educational Research and Development, 1855 Folsom 
Street, San Francisco, California 94103 



Bela Banathy, Far West Laboratory (415) 565-3197 



Vz\)QJiopmzvit 



A TECHNOLOGY FOR DEVELOPING INSTRUCTIONAL MATERIALS 



This self-instructional program provides training for developers of 
Instructional materials. Trainees engage in active practice of all 
the major tasks in the instructional development process, including: 
task analysis, stating objectives, developing tests, planning 
simulations, formulating strategies, developing materials, trying 
them out, and revising them. A Handbook is available to serve as an 
aid during learning, as a job aid immediately after program completion, 
and as a reference source as the developer gains experience. Active 
practice of small development steps is provided in an accompanying 
workbook. Active practice of large tasks (combining the small steps) 
is provided in a final exercise volume. Final student evaluation 
exercises are included in the materials. 



toKQoX Auctcence: 



y^atzJiicUU and 



lyUtAuctiondi 
AvcUlabUAXy* 



Beginning and intermediate audiences; 
portions for advanced audiences. 

30-50 hours of self-paced instruction 
for use in academic, industrial, or 
military settings. 



Printed materials include Handbook, with 
11 subvolumes. Workbook, Final Exercise, 
User's Manual, and Orientation. No 
equipment required. 



None 

Currently in trial and revision stages. 
Scheduled for publication within a year. 

Not specified 

George L. Gropper, American Institutes for 
Research, 710 Chatham Center, Pittsburgh, 
Pennsylvainia 15219 (412) 281-1100 



Not specified 



TO 

Inion,mcCtLon Vi^^mlnoution/ 
CormuyilcatLon 



THE EDUCATIONAL INFORMATION CGUNSULTANT: 
EDUCATIONAL INFORMATION 



SKILLS IN DISSEMINATING 



The EIC package provides skill development training for a role in education 
of a "middleman" who knows how to find and make accessible educational 
information where it is needed. The program is based on a model which 
delineates five major processes for this role. Participants learn to: 
analyze and define an educator client's information problem; 
execute an information search; screen, sort, and package the 
communicate it to the client; and evaluate how effective the 
been. The training is skills-oriented and learner-centered, 
feature group and individual activities, including 
solving, simulation, and decision making. Student 



plan and 
information; 
service has 
The exercises 
role playing, problem 
self-evaluation 



exercises and skill evaluation criteria and devices are included in the 
package. The training is available in three forms: Course, Institute, 
and Learning Team. 



Ta/iQzt Audizncz: 



^^oteAlaJU and 



ln6t/iucitionaZ 



Educational information dissemination personnel at 
school, district, county, regional, or state levels, 
e.g., information specialists, librarians, curriculum 
or subject matter specialists, instructional materials 
center personnel, graduate students. 

For preservice or inservice training as a 45-hour 
graduate course, as a 10-day workshop, or as an in- 
dependent study program for a team of three. 



Training Manual (one per student) and an Instructor's 
package, with Instructor's Guide, model information 
packet, communication game, two filmstrips, and two 
cassette audiotapes. Filmstrip projector and cassette 
tape recorder. Access to an ERIC collection and to a 
library or information center. 



Course and Institute can be directed by any university 
instructor; a background in information utilization is 
helpful, but not required. Learning Team is self- 
administered. 



KvcUZoibltiXij: 
Developer: 



Course and Institute as of April 1 , 1973; field test 
version of Learning Team as of March 15, 1973. 

Training Manual, about $10; Instructor's package aboui $60. 

Bel a Banathy, Far West Laboratory for Educational Research 
and Development, 1855 Folsom Street, San Francisco, 
California 94103 (415) 565-3197 



VU tn,ibut0K: 



For Course and Institute, Bela Banathy. For Learning 
Team, Dept. B, University Extension, University of 
California, Berkeley, California 94720 
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Local Changz/ 
Vi^iulon 0(5 JnnovcuUoyib 



TRAINING FOR LEADERSHIP IN LOCAL EDUCATIONAL IMPROVEMENT PROGRAMS 



The program trains personnel for a specialty in designing, conducting, 
evaluating, and diffusing a local change program. Ten modules cover 
the following aspects: problem-solving model for designing and 
conducting local improvement programs, working relationships in 
leadership for change, change theory and strategies, educational reform 
themes and related innovations, implementing innovative programs, and 
diffusing innovations within school systems. A module can be studied 
independently of the rest of the program; some modules are prerequisites 
to others. Each module contains an explicit statement of training 
objectives, a study guide, exercises based on simulated or real 
situatipns, and an achievement posttest. Although the modules are 
self-instructi6nal , it is recommended that an instructor be available 
to assist trainees with problems encountered and to assess accomplish- 
ment of objectives. Directions for assessing products of trainee 
practicum exercises are also included. Access to an educational 
library, ERIC, and other reference sources is important. In addition, 
access to educational R&D agencies and school systems is necessary for 
practicum experiences. 



ToAget Audience 



Mate/iiaJU and 
Eqitipmnt Hztdzd* 



iMtAacZionai 
Availabltittj: 

VavzZopnx: 

V.a>Vi.Lbutox: 



School district administrators and specialists 
and field consultants in state education depart- 
ments, educational labs and centers, schools of 
education, or private agencies. 

Self-instructional modules for use in inservice 
or preservice training programs. Each module 
requires from 6-15 hours to complete. 



Ten self-contained modules, plus some required 
and optional readings. Audiotapes and tape 
recorder. 



Instructor should be available for consultation 
and student assessment. 

From developer after July 1973 and from 
publisher by January 1974. 

Estimated $10 or less per module. 

Glen Heathers, Director of Staff Development, 
ACP, Research for Better Schools, 1700 
Market Street, Philadelphia, Pennsylvania 19103 
(215) 561-4100, Ext. 232 

Publisher not yet selected. 



ERIC 
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Contaxt 
EvcUbicution 



H A SEMINAR AND TRAINING PROGRAM IN NEEDS ASSESSMENT AND GOAL DEVELOPMENT 



Two sets of complementary materials are provided for training on needs 
assessment and goal development in local school districts. The first is 
a Training Program for school-community groups on setting goals for 
their school, based on comprehensive needs assessment. It consists of 
eight modules, accompanied by materials for a trainer. The trainer is 
allowed some flexibility in arranging the materials for adaptation to 
the local situation. The second set of materials takes the form of a 
university-level seminar for training professional-level developers and 
evaluators in similar skills on a more sophisticated level. These 
materials also have a modular format. Both sets make extensive use of 
simulations, tape/slide presentations, and self-instructional exercises. 
Self-evaluation exercises on the content of each module are provided. 



Local school staff, parents, and community 
residents (Training Program). Educational 
administrators, program developers, and 
evaluators (Seminar). 



16-20 hours for each program. 



MateJuatd and 
Equiprnzrvt Hzddzdi 



Modules, cassette tape, cassette tape recorder, 
overhead projector, screen. 



InstAuctioytat 
RzquAAmznts: 



One trainer per 12-14 participants. 



Fall 1973 



To be determined 



VzvztopeAi 



Dr. James M. Morgan, Cincinnati Public Schools, 
230 East Ninth Street, Cincinnati, Ohio 45202 
(513) 621-7010, Ext. 293 



None yet selected 



ERIC 
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ll^ban Ediiccvtion 

H THE FORD TRAINING AND PLACEMENT PROGRAM 

The Ford Training and Placement Program is a cooperative effort between 
the University of Chicago and the Chicago Public Schools to develop and 
test a social systems model of training teachers and others for urban 
schools- The basic training unit, a cadre, is composed of preservice 
interns, experienced teachers, administrators, psychologists, community 
representatives, etc. They train together during the internship year. 
Interns are committed to full-time employment during the following year 
at the school. The program staff selects, trains, and evaluates each 
group. The program provides a model for training professionals for 
inner-city schools and includes instruments, program descriptions, 
and consultation service. 

ToAgoX Aactcence: Experienced teachers and students in elementary 

and secondary education. 

Suggutzd U6z: Thirty graduate hours, if structured as a degree 

program for experienced teachers; three years, 
if structured as an undergraduate program; one 
year beyond subject area courses, if structured 
as a preservice graduate program. 

lAdtzHloJU and 

EqtUpmznt Hzzdzd- Wide variety of literature on inner city schools 

and group process training materials. Micro- 
teaching clinic equipment and portable videotape 
equipment useful, but not essential. Cooperative 
arrangements with inner city schools for observa- 
tion and training purposes. 



Full-time director with experience in the schools 
and the university; cadre 1 iaisons— half-time 
person for each training group during internship 
year; group process consul tant--half-time person 
for each training group; and full-time evaluator. 

1973 

Program costs, depending on structure and tuition, 
run from $2,000 to $10,000 per year per training 
group. Fixed staff costs run about $15,000 
per year. 

Henrietta Schwartz, Director, Ford Training and 
Placement Program, Graduate School of Education, 
University of Chicago, 5835 Kimbark, Chicago, 
Illinois 60637 (312) 753-3741 

Same as Developer 



iMiMiQjtLonaZ 



Kvaitabitityt 



V<6 txibuZon.'* 



ERIC 
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Diffusion 



Training for Change Agents: A Guide to the Design of Training Programs 

in Education and Other Fields 
Havel ock, Ronald G. 
Ann Arbor, Michigan 
The University of Michigan 
1971 

This volume is largely based on the Conference on Educational Change 
Agent Training which included members from public schools, college and 
university education departments, educational research centers, and 
state and federal educational agencies. Its purpose is to provide 
trainers and training program developers with guidelines for effective 
use of training programs and of available resources. The conference 
members initially discussed various research findings and theory which 
they believed to form a sound basfs for training models in the change 
process. Four specific goals of training and fifteen principles of 
good training design are developed and defined. Based upon these goals 
and principles, a comprehensive outline consisting of eight elements is 
developed and described to serve as an evaluative checklist for training 
programs. The conference meiribers then divided into groups to use the 
criteria developed to formulate various types of training program models. 
The results are presented in outline form and cover the following areas: 
training school systems to develop a self -renewal capacity; producing 
structural and political changes in school systems; linking school systems 
with available reso'jrces; and improving the effectiveness of other educa- 
tional agencies. Also presented is a fully developed, ideal model program 
for state educational agencies to train in change planning and management 
skills. 

Performance Evaluation 



Analyzing Performance Problems or *You Really Oughta Wanna' 

Mager, Robert F., Pipe, Peter 

6 Davis Drive, Belmont, California 94002 

Fearon Publishers/Lear Siegler, Inc., Education Division 

1970 

This volume presents a detailed procedure for identifying and analyzing 
the nature, importance, cduse, and possible solutions of performance 
problems, i.e., when an individual's actual performance differs from the 
desired performance. Although the technique is applicable to everyday 
life, the volume is basically oriented towards educational and industrial 
environments, from which numerous examples are used. Utilizing a flow 
diagram and a list of appropriate questions, the first steps are to de- 
termine the nature and importance of the performance discrepancy and 
whether it is due to an actual skill deficiency. If a skill deficiency 
exists, one then determines: 1) whether the skill once existed; 2) whether 
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the lost or deteriorated skill Is used frequently or infrequently; 
3) whether there is a solution simpler than performance maintenance or 
fomal training; and 5) whether the person has the potential to perform 
the skill. If a skill deficiency does not exist, one determines: 1) 
whether desired performance leads to unfavorable consequences; 2) whether 
non-performance or other performance leads to more favorable consequences; 

3) whether there is a meaningful consequence for desired performance; and 

4) whether there are obstacles to the destred performance. Proposed 
solutions are presented for all the above possible causes of performance 
problems. Finally, one must compare the size of the remedy with the size 
of the solution. In conclusion, a checklist of key issues and appropriate 
questions for analyzing performance problems is summarized. 

Research and Development 



Procedures for Conducting an Apprenticeship Work -Study Program 

Millstein, Eugene J., Melnotte, Judith M., Dunn, James A. 

Palo Alto, California 

American Institutes for Research 

1972 

This manual describes procedures for conducting a cross -training and 
apprenticeship program for individuals interested in a career in educa- 
tional research and development. It is intended as a guide for those 
interested in administering such a program and is based on the experiences 
of a pilot program conducted by the American Institutes for Research. The 
program description contains information on: length of the program; site; 
target population; instructional system; curriculum; recruitment procedures; 
selection criteria; stipend; and assistance with job placement. The follow- 
ing information on the instructional model is then provided: curriculum 
outline; how to identify individual objectives, prescribe learning activi- 
ties, and monitor trainee progress; and how to select, assign, and monitor 
apprenticeship tasks. Publicity, application, and selection procedures, 
and program management considerations are then discussed. The importance 
of job placement activities if the program is used as an independent train- 
ing program or as an adjunct to regular academic training is discussed. 
Appendixed are a list of proposed instructional objectives, a bibliography 
of instructional materials and resources, and a sample fact booklet for 
prospective applicants. 
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Communication Skills 



So You Have to Write a Technical Report: Elements of Technical Report 

Writing 
Gray 

This volume, written for anyone who has to write a technical report, is 
primarily concerned with those aspects and principles which are applicable 
to the preparation of all technical reports. The work's objectives are to 
provide basic guidance in identifying the audience for the report, organiz- 
ing material and planning the document, and writing the full report. After 
defining a technical report, its principal function, characteristics, and 
various types of technical reports are detailed. Technical report writing 
techniques are identified and discussed and their advantages and limitations 
are pointed out. The initial steps of report writing discussed are analyzing 
the overall problem, assembling necessary data, preparing a working outline, 
and thinking about the report presentation. The parts of a report are de- 
scribed in the order in which they should be written; 1) introduction; 
2) main body; 3) conclusions and recommendations; 4) appendixes, references, 
footnotes, and tables of contents; 5) abstract; and 6) title page. For 
each report part, its function, material to be included, location in the 
report, possible formats, style, and organization are discussed. The 
following characteristics of good technical report writing are then considered: 
completeness, conciseness, veracity, restraint, clarity, and ^general ap- 
pearance. Finally, a summary of the volume is detailed in outline form. 

Develppment of Programmed Instruction 

Developing Programmed Instructional Materials, A Handbook for Program 
Writers 

Espich, James E.. Williams, Bill 

2165 Park Boulevard, Palo Alto, California 94306 

Fearon Publishers, Inc. 

1967 

This volume briefly discusses the theories behind programmed instruction, 
describes the preparatory steps required for programming, details the ma- 
jor prograrrming techniques for writing various types of test frames, and 
suggests procedures for editing, testing, and test analysis. Although 
mainly an introductory text for program writers, it is also oriented 
towards individuals involved in evaluating programmed instructional ma- 
terials. Theories of behavioral psychology are discussed as the basis 
for programming techniques. Three basic questions are then detailed which 
should be posed to determine the feasibility of wirting a program. The 
progrirjmmer then must: establish objectives; examine the scope, level and 
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results of material currently utilized; discuss the program with a subject 
expert; and diagram the material to be presented using flow charts or 
schematics. The following program construction techniques are discussed: 
discrimination frame sequence; constructed response frame sequence; branch- 
ing frame sequence; retrospective chaining; BABOON frames; adjunct program- 
ming; and adjustive techniques such as branching programs, gate frames, 
remedial loops, and secondary tracts. Editing for programming techniques, 
technical accuracy and composition are discussed. One-to-one, small group, 
and field testing are discussed as experimental testing techniques. After 
testing, the minimum effort required to comprehensively revise a program 
is detailed. 



* * * * 



How To Write A Program, A Programmed Course Designed to Teach Techniques 

for Programming Instruction 
Silverman, Robert E. 
Box 156, Carlisle, Massachusetts 
Carlisle Publishers, Inc. 
1970 

This programmed instruction text is an untheoretical introduction to how 
to write programmed materials* Designed for teachers or trainers with no 
knowledge of programned instruction, the reader learns to apply principles 
derived from the psychology of learning and programming experience. The 
text's objectives are to teach the following: 1) to state objectives in 
terms of desired student responses; 2) to use prompts and proper sequence 
to obtain the desired responses; 3) to use positive, immediate feedback; 
4) to require relevant and effective responses; 5) to use RULEG, EGRUL, 
discrimination, and response-familiarization sequences; 6) to write teach- 
ing, practice, review, and test frames; and 7) to use branching, a technique 
which provides individualization within a program. Instruction and practice 
are provided for each of these basic programming skills and techniques. In 
conclusion, an exercise is provided which is designed to utilize the reader's 
newly-learned skills by actually writing a programmed lesson. The text 
also includes a glossary of technical terms and a bibliography of suggested 
readings. 
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Practical Programming 
Pipe, Peter 

Holt, Rinehart and Winston, Inc. 
1966 

This volume is designed to help start the beginner in writing a program; 
it covers practical issues such as preparatory steps, programming tech- 
niques, and testing and analysis methods. It is written for anyone who 
has decided that he is going to write a program. A brief introduction to 
programmed instruction defines four of its major characteristics. Also 
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discussed are the characteristics, advantages and disadvantages of the 
two major programming approaches;;, linear programming and branching pro- 
gramming. Six steps necessary for program writing preparation are de- 
tailed and discussed. In actually writing the program, its general format 
and such issues as introduction content, amount of practice necessary, and 
when to summarize are discussed. The written program should contain the 
following discussed components: 1) introduction; 2) review of essential 
concepts assumed as prerequisites; 3) step-by-step development of new con- 
cepts; 4) practice; and 5) final sumnary and criterion test. Numerous 
techniques and considerations in designing branching and linear programs 
are discussed using diagrams and examples. Finally, techniques for program 
testing, revision, and editing are briefly described. 

Diffusion 

A Guide to Innovation in Education 
Havelock, Ronald G. 
Ann Arbor, Michigan 

The University of Michigan, Institute for Social Research 
1970 

This manual contains information about how change occurs and how educators 
who work for change (change agents) can organize their work to make suc- 
cessful innovation tak^ place. The orientation is toward problem solving 
by and for the user through effective use of resources. Material included 
is based on an analysis of 1,000 studies of innovation and knowledge utili- 
zation in education and other fields. The process of change and innovation 
is divided into six stages: 1) building a relationship {between change 
agent and client); 2) diagnosing the problem; 3) acquiring relevant resources; 
4) choosing the solution; 5) gaining acceptance; and 6) stabilizing the in- 
novation and generating self-renewal. The first part of the Quide gives 
case studies of projects in school settings to illustrate the process. Part 
two, the main text of the manual illustrates the six stages of planned change 
with quotes from prominent authors and references to the case examples. 
Supplementary resource information is covered in the third part of the manual 
and includes sections: A. strategies and tactics: a glossary and guide to 
selection, which goes over conmon change strategies and tactics; B. major 
information sources in education which gives bibliographic and annotative 
services, directories and indices; consulting organizations; and government 
agencies; and C. a guide to the literature on planning of change in educa- 
tion with annotations and author and subject indices. The guide is arranged 
for easy reference. 

Information/Data Collection 



Doing a Literature Search in an Information Center 

Garman, Marcia B. and Kathleen Mitchell 

Educational Resources CSYJter 

590 Hamilton Street 

Redwood City, California 94063 

This course is intended to teach individuals who want to become educational 
information consultants different methods of doing a literature search in 
an information center. Through numerous practice exercises, activities, 
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readings, tapes, slides, and films, the reader is introduced to both tra- 
ditional and nontraditional literature sources. The course's objectives 
are: 1) to identify various relevant glossary terms; 2) to transform a 
given search request into applicable descriptor terms; 3) to use manual 
techniques to select ten relevant documents, for a given search request, 
from ERIC publications and "fugitive" information catalogues; and 4) to 
use computer techniques to select ten relevant documents, for a given 
search request, from DIALOG, an on-line retrieval system for educational 
materials. Numerous se'if-tests are included, with their answer keys for 
the instructor. Equipment, materials, and facilities needed are listed. 

Instructional Objectives ^ 

Preparing Instructional Objectives 

Mager, Robert F. , Ph.D. 

6 Davis Drive, Belmont, California 94002 

Fearon Publishers/Lear Siegler, Inc., Educational Division 

1962 

This volume utilized programmed instruction to explain how to specify 
and communicate instructional objectives in concrete, evaluative form. 
It is written for teachers and student teachers in all subjects and at 
all levels who are interested in preparing auto-instructional materials. 
The book's objectives are to teach the reader to recognize instructional 
objectives which are stated in terms of performance, to identify those 
portions which define acceptable performance, and to select from various 
test items those which appropriately evaluate the stated objectives. The 
importance and advantages of usefully stated objectives are discussed 
using numerous examples. The following qualities of meaningful objectives 
are described: identifying the desired behavior; specifying the condi- 
tions under which this behavior should occur; and specifying the criteria 
of minimally acceptable performance. A self- test is provided to determine 
the reader's ability to identify the characteristics discussed. Supple- 
mental test is also included but not necessary to achieving the document's 
objectives. There is no discussion of the philosophy of programmed in- 
struction, who should select objectives, or which objectives should be 
selected. 

Measurement and Evaluation ^ 



Program Evaluators Handbook, Measurement 

Farr, S. David, Subkoviak, Michael J. 

Albany, New York 12224 

The State Education Department 

1969 

This manual is the result of a review and training session for strength- 
ening evaluative techniques. Conducted by the State of New York, the 
session was specifically concerned with Title III of the Elementary and 
Secondary Education Act of 1965, although the material is general izable. 
The material is presented such that the original training session is 
replicable with time devoted to lecture, class discussion, and practice 
exercises. The manual has five units with expository text, an outline 
for use in lecture and/or class discussion, and exercises listed for each 
unit. The five unit topics are: 1) measurement-purposes, ideals, possi- 
bilities; 2) defining measurement domains, 3) person and item sampling; 
4) test and item selection; and 5) objective observation. A Selected 
List of Standardized Tests details each test's name, publisher, grade 
levels covered, testing time, major subject areas measured, and where it 
is reviewed in the Mental Measurements Yearbook. 
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A Programned Introduction to Educational and Psychological Measurement 

Levine, Samuel and Freeman Elzey 

Brooks/Cole Publishing Company 

Belmont, California 

1970 

Using a programmed instruction approach, this volume discusses the basic 
concepts of measurement as they relate to test development and interpre- 
tation. The work is designed to supplement texts in measurement and 
evaluation in the behavioral sciences and Is written for individuals who 
have had an introductory course in statistics. The following topics are 
discussed: 1) the differences between qualitative and quantitative 
measurement and between discrete and continuous variables; 2) nominal, 
ordinal, interval, and ratio measurments; 3) adequately defining the 
phenomenon to be measured; 4) concurrent and predictive validity; 5) the 
Standard Error of Measurement; 6) test reliability and methods for de- 
termining the degree of stability in test scores over a short time period; 
7) the use of equivalent tests to determine reliability; 8) the internal 
consistency of a test using such tTieasurements as the coefficient of sta- 
bility, the coefficient of equivalence, the coefficient of internal con- 
sistency, and the Spearman-Brown Prophecy Formula; 9) using a correlation 
matrix to determine which test items to delete or retain; 10) difficulty 
level and discriminating power for item analysis and the rationale and 
techniques for computing the Ease Index and the Discrimination Index; 
11) ranks and percentiles; 12) test norms; and 13) use and interpretation 
of 2 scores, t scores, stanines, and deviation IQ's. 



★ ★ ★ ★ 



What Are Students Learning?, Documenting Multiple Effects of Instruction 

on Learners, Booklet 1 
Rosen, Marvin J. 
Durham, North Carolina 27701 
National Laboratory for Higher Education 
January, 1972 

This booklet describes a series of procedures for specifying concrete 
objectives and briefly discusses the measurement process. The following 
three types of information in a well -formed objective are described: 1) 
outcome formation, or the behavior desired; 2) level of achievement for- 
mation, or the specification of minimal acceptable performance; and 3) 
circumstances of evaluation formation, or conditions under which perfor- 
mance will be evaluated. The differences between general and specific 
objectives and refining the general objective into specific objectives 
are discussed. A form for refining general objectives so that they 
clearly and unambiguously define the skills, knowledges, and attitudes 
to be acquired is provided. The three major classes of learning, psy- 
chomotor, cognitive, and affective, are defined and exemplified. Methods 
are discussed for identifying and classifying these types of learning by 
level of difficulty or complexity. Finally, the necessity of specific 
procedures for verifying, testing, observing, or noting student achieve- 
ment of specific objectives is briefly discussed. 
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Which Students Achieved Which Objectives? Summary Statistics for 

Documenting Criterion-Referenced Instruction, Booklet 2 
Rosen Marvin J. 
Durham, North Carolina 27701 
National Laboratory for Higher Education 
January, 1972 

This booklet details a procedure for obtaining evaluation statistics on 
criterion-referenced instruction, which is instruction designed to accom- 
plish certain specific objectives. The procedure helps to describe clearly 
the skills mastered and missed by students. The reader should learn to 
perform the following tasks: 1) given one or more test items for an ob- 
jective and a criterion of successful performance, calculate the percentage 
of students achieving the criterion; 2) given objectives with varying cri- 
teria for successful performance, calculate the percentage of students 
achieving the criterion for each objective; and 3) given criterion levels 
of successful performance for objectives and the percentage of students 
achieving these levels before and after instruction, display in tabular 
form any discrepancies and the effectiveness of the program relative to 
each objective. A form is provided for tabulating the achievement of 
learning criteria which is useful in pretests to determine: to exclude 
individuals from doing an instructional sequence for a particular objec- 
tive if their scores indicate that they have achieved the criterion; and 
areas to be emphasized for those who fail to achieve the criterion. It 
is useful in posttests to determine: the students who achieve the criterion 
after, but not prior, to instruction; and if revision of the instructional 
sequence is necessary. 
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Program Evaluation , 

Assessement and Evaluation Handbook, Title I 
Finkelstein, Elsie L. 
Albany, New York 12224 

The State Education Department, Office of Research and Evaluation 

This manual basically presents an overview of the evaluation process \ 
and extensively covers numerous evaluation planning charts and assessment 
devices for the evaluation of specific programs. It was developed by the 
New York State Education Department and is intended as a guide to assist 
local school personnel in assessing the effectiveness of ^prp.grarns and_ . 
projects initiated under Title I of the Elementary and Secondary Education ^ 
Act of 1965. Initially discussed are the purpose of evaluation, the plan- 
ning and development of evaluation procedures, and six basic evaluation 
designs. Utilization of the State Pupil Evaluation Program and state 
agencies available for assisting project personnel are briefly described. 
Twenty-nine samples of evaluation planning charts are then included for 
use as guides or suggestions when planning evaluation procedures. Each 
chart has four headings for listing the general objective, specific ob- 
jectives, pertinent behavioral criteria, and evaluation procedures. The 
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sample charts are on the following topics: educational communications; 
general education; handicapped children; health, physical education, and 
recreation; humanities and the arts; industrial arts education; occupa- 
tional education; pupil personnel services; school supervision; and 
teacher education. Several of the sample charts include examples of 
various relevant evaluation devices such as questionnaires, checklists, 
etc. An annotated bibliography is included which lists numerous source 
materials pertaining to school program evaluation. 



* * * * 



Program Evaluators Handbook, Sampling Model 

Meyer, Donald 

Albany, New York 12224 

The State Education Department 

1969 

A Bayesian model for determining a variable's confidence interval, given 
there were random and nonrandom sampling errors, is presented in this 
booklet. Initially, a standard formula is presented for determining a 
confidence interval when there is random sampling error. The following 
types of sampling errors are then discussed: frame error; selection 
error; nonresponse error; measurement error; and random sampling errors. 
Given an example problem with estimated error values for the above error 
parameters, a tree diagram is utilized to perform a population analysis 
and to show the proportion of the various error sources. The possible 
dependent relationship among the error parameters may be eliminated by 
the formulation of new parameters which are ratios of the original error 
parameters. A formula is given for this error ratio model which incor* 
porates formulas for frame, selection, random sampling, non-response, 
and measurement errors. The ratio model is then used to perform a total 
error analysis for an example problem. 



* * ★ * 



Urban Education Program, Evaluation Manual 

The University of the State of New York 

Albany, New York 12224 

The State Education Department 

June, 1969 

This manual is concerned with the evaluation of projects in the State 
of New York's Urban Education Program , which has two priority areas: 
1) projects relating to the development and operation of comnunity edu- 
cation centers; and 2) projects directly related to the regular elementary 
and secondary school programs. The manual is designed to aid school dis- 
tricts in developing and implementing effective evaluation programs for 
these projects. The following basic components of the evaluation plan are 
discussed: environmental conditions; objectives; project activities; 
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evaluation design; and decision processes. Model evaluation plans are 
exemplified in the following areas: school readiness; reading achieve- 
ment; mathematics achievement; guidance services; retention of dropouts; 
and effects of cultural deprivation. Each plan states its general goal 
and specific objectives and each objective lists the target goal perfor- 
mance and evaluation procedures. The following information is provided 
on various types of evaluation instruments for target group performance: 
description; when they may be used advantageously; types of information 
they should supply; brief guidelines for their design; and good and poor 
examples. An outline which should be used in the development of project 
evaluation reports is detailed. Finally, appendixed are a list of stan- 
dardized measurements for basic skills educat^^on, samples of evaluation 
instruments, and an annotated bibliography of selected references in 
evaluation. 

Program Research and Development 

Improving the Program, Revising Instructional Materials in Light of 

Tryout Data, Booklet 5 
Rosen, Marvin J. 
Durham, North Carolina 27701 
National Laboratory for Higher Education 
1971 

This booklet describes a procedure for analyzing student responses to 
identify the types of deficiencies in an instructional program and to 
determine specific, remedial revisions. Student responses are to be ob- 
tained from criterion-referenced tests, practice exercises, and interviews. 
To locate the program deficiencies, a flovf diagram for their analysis de- 
tails all the stops which should be examined before determining a possible 
program revision. First, the defective element should be identified and 
determined whether it is sufficiently important to warrant a revision effort. 
If it is not very important, the criterion for acceptable performance might 
be lowered or the associated instructional materials could be eliminated 
from the program. If the material is important but the student is not per- 
forming tne learning activities associated with it, it should be determined 
whether: 1) the students know what they are supposed to do; 2) they are 
trying; 3) they have the materials necessary; and 4) there are 
other obstacles to performance. If the student is performing the learning 
activities, but unsuccessfully, it should be determined if the criterion 
measures are valid and if the students are making errors when practicing. 
For each of the above possible problem areas, remedies for their solution 
are briefly discussed. 



★ ★ ★ ★ 



Instructional Product Research 

Baker, Robert L., Schutz, Richard E., ed. 

11300 La Cienega Boulevard, Inglewood, California 90304 

Southwest Regional Laboratory for Educational Research and Development 

1972 

The material in this book presents various research tools and techniques 
Q relevant to the research process and useful for developing instructional 
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products. It is intended for the user of research and for novice in edu- 
cational research and development. Initially discussed are the classifi- 
cation of variables, the use of status, association and experimental 
studies, and interpreting the results of educational research studies. 
Identifying and selecting dependent and independent variables, and basic 
problems to avoid in their selection are discussed, A brief discussion 
of the research problem and guidelines, and sources for reviewing relevant 
literature are provided. In preparing a research proposal, the reader 
learns to outline the requirements and conditions of its basic components: 
the title, the research problem, procedures, and logistics. The inter- 
rupted time series design and the following control group designs are dis- 
cussed as being very suitable for evaluative investigations in schools: 
nonequivalent, pretest-posttest, and posttest only. A strategy is provided 
for choosing appropriate statistical procedures for analyzing research data 
and the reader is familiarized with computer terminology and the use of 
library computer programs for statistical anlaysis. Formulating the re- 
quirements and conditions of the following research report components are 
discussed: title, research problem statement, method, results, discussion, 
conclusion, $ummary and abstract. 

Research and Evaluation 



An Evaluation Guidebook, A Set of Practical Guidelines for the Educational 

Evaluator 
Popham, W. James 

Box 24095, Los Angeles, California 90024 
The Instructional Objectives Exchange 
1972 

This volume brings together arid describes a collection of relevant, re- 
cently developed, technical procedures which may be used as a practical 
set of guidelines for the educational evaluator. After defining educa- 
tional evaluation, its two main roles are described: identifying the 
instructional objectives and judging the quality of the instructional 
procedures designed to attain the objectives. The guidelines for instruc- 
tional objectives concern: 1) the role of measurability; 2) unmeasurable 
objectives; 3) acceptable performance criteria; 4) content general vs. 
test item equivalent objectives; 5) minimal proficiency levels; 6) taxonomic 
analysis of objectives; and 7) utilizing existing collections of objectives. 
Guidelines for measurement examine: •!) criterion-referenced measures; 2) 
domain-referenced achievement testing; 3) multiple criterion measures; 4) 
unobstrusive measures; and 5) unanticipated outcomes. The guidelines for 
data collection and analysis concern: 1) clarifying value preferences; 
2) comparing preference and performance data; 3) person and item sampling; 
4) formative evaluation; 5) summative evaluation; 6) appropriate units for 
data analysis; 7) estimation vs. hypothesis testing; and 8) cost/effective- 
ness decision making. Each of the above guidelines suggest a practical 
course of action for the educational evaluator. Finally, an example is 
presented of an evaluator who utilizes the guidelines. 
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Handbook in Research and Evaluation 

Isaac, Stephen 

Robert Knapp, Publisher 

San Diego, California 92107 

1971 

This handbook is a collection of principles, stategies and methods which 
are useful in the planning, design, and evaluation of research in the 
benavioral sciences and education. Using almost a checklist approach, 
the handbook is designed to present an overview, a summary of alternatives, 
or a listing of strengths and weaknesses of various techniques for behav- 
ioral research. The volume is intended as a general reference tool for a 
research team, project director, proposal writer, evaluator, etc. Initially 
presented are several sets of checklists and guidelines which should prove 
useful in research planning. As resear*ch design depends upon the purpose 
and nature of the. study, nine design alternatives are briefly described. 
Also, various possible problem areas which the research must consider such 
as validity, regression effects, etc. are detailed. Various techniques 
for establishing measurable criteria for research are discussed. Briefly 
summarized next are the more commonly used statistical techniques and 
guidelines are provided for their use. Finally presented are guidelines 
for planning, preparing, writing, and evaluating a research proposal, 
thesis, report, or article. 

Statistics 



A Programned Introduction to Statistics, 2nd edition 

Elzey, Freeman F. 

Belmont, California 94002 

Wadsworth Publishing Company 

1971 

This book is designed to introduce one to statistical techniques and their 
application through programmed sets of instruction. It is divided into 
twenty-five sets, each containing a) an introduction; b) the programmed 
instruction portion; and c) a series of exercises. The introduction of 
each set states the objectives of that chapter; the programs are divided 
into frames, each requiring an answer to a fil 1-in-the-blank statement; 
the exercises provide an opportunity to test what the sets have presented. 
Formulas, tables, and a glossary of statistical symbols are provided. 
Topics covered include: organization of data, measures of central ten- 
dency (mode, median, mean), comparison of measures of central tendency, 
percentiles, graphic representation of frequency distribution, measures 
of variability (range, semi -interquartile range, average deviation, vari- 
ance, standard deviation), relationship between population and sample, 
introduction to the normal distribution, probability and the normal dis- 
tribution, sampling error, estimation of population variance, estimation 
of the standard error of the mean, 95% confidence interval, 99% confidence 
interval, t^ distribution, null hypothesis, the standard error of the dif- 
ference between means, t^ ratio for independent and non-independent means, 
one-and-two tailed tests, type I and type II errors, analysis of variance, 
£ test for two variance estimates, scatter diagrams, introduction to cor- 
relation, Pearson product-moment correlation, regression. Spearman's rank 
order correlation, chi square (single sample and multiple samples). 
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Curriculum Research and Development 



Recommendations for Curriculum and Instructional Materials 
Tyler, Louise L., Klein, M. Frances, Michael, William B. 
2265 Prosser Avenue 
Los Angeles, California 
1971 

Twenty-eight recommendations for identifying significant aspects of in- 
structional materials are proposed as possible criteria for determining 
the materials' quality and for providing a basis for making judgments re- 
garding them. These recommendations are intended for producers of curri- 
cula and instructional materials, for users in their selection, and for 
funding agencies in their evaluation. The recommendations concern the 
following aspects of materials: rationale, appropriateness, specifications, 
effectiveness, conditions, practicality, and dissemination. After stating 
the recommendation, it is designated as essential, very desirable, or de- 
sirable. Each recommendation is then discussed in relation to several of 
the following terms: rationality, values, decisions, accountability, 
significance, comprehensiveness, causation, behavioralism, and knowledge. 
The recommendations are then analyzed and evaluated within the framework 
of a specific teaching model, first proposed by Glaser, which consists of 
four major components: objectives; entering behavior; instructional pro- 
cedures; and the evaluation of outcomes. The evaluation of materials and 
the components of the evaluation process are discussed in regard to de- 
termining whether the recommendations have been utilized. Finally, a 
transcript of a meeting with the authors and a group of teachers concerned 
with selection of materials is included. 
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Research for Curriculum Improvement 

Association for Supervision and Curriculum Development 

NEA 

This volume, which is the 1937 yearbook of the Association for Supervision 
and Curriculum Development of the NEA, provides a history of curriculum 
research, describes the research process, discusses the researcher himself, 
and details various aspects of cooperative curriculum research in the 
school setting. The work is intended to fulfill the needs of all educators 
considering, or actual ly doing, research in instructional improvement. 
Initially provided are a discussion of the theoretical principles which 
must be considered by curriculum researchers and a.. history of curriculum 
research. Section II covers the phases of the research process: 1) 
problem identification; 2) determining working hypotheses; 3) selecting 
and handling data; and 4) obtaining meaningful results from the data. 
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Section III asks the reader to consider himself as not only a consumer, 
but also a producer, of research; and discusses the necessity of good 
human relations when conducting research. The final section discusses 
the following aspects of conducting curriculum research in the school: 
organization and management of the research process; inhibitors in con- 
ducting research; and ethical considerations when using students in edu- 
cational experiments. Appendixed are an annotated bibliography on 
curriculum research and a report on the first Cooperative Curriculum 
Research Institute. 

Information/Data Collection 



A Guide to Educational Resources 

Far West Laboratory for Educational Research and Development 

Intended primarily for the educational information consultant, this brief 
annotated bibliography is designed to selectively cover standard sources 
of information for preliminary literature searches and those literature 
tools and services available for current awareness of activities, products, 
sources and innovations. The full bibliographical citation and the work's 
price are detailed, when appropriate, for each item covered. The work 
describes reference toois available for: general fact finding; social 
science sources; guides to educational literature and research; basic 
reference in education; statistics; directories; guides to current litera- 
ture. Office of Education projects, funds, and selection of periodicals; 
and guides for products and audiovisual materials. Information is pro- 
vided on the Educational Resources Information Center (ERIC), its twenty 
clearinghouses, their mailing address, and areas of specification. Also 
listed are the mailing addresses of regional laboratories, university- 
sponsored research and development centers, and several related agencies 
which are funded by the U.S. Office of Education. Several nonconventional 
information services for current awareness and retrospective literature 
searches are also annotated. 

Planning and Management 

Administrator's Manual for State Plan Programs, Title III, 

Elementary and Secondary Education Act. 
U. S. Department of Health, Education, and Welfare 
Office of Education 

Projects to Advance Creativity in Education 
1971 

This manual attempts to provide a base of common understanding among 
State educational agencies on the legislative changes occurring in the 
Title III Program since 1*969. Changes In Title III are listed and ex- 
plained. The main text of the manual centers on the details of state 
plan administration. Detailed information is provided on eligibility 
for Federal funds. State organization and functions (State advisory 
council and State agency), and the State plan program. Discussion on 
the State plan program covers the details of the State educational agency 
as facilitator of educational change, assessment of educational needs, 
project development and proposal review, evaluation of projects, dis- 
Q semination and change strategies, guidance, counseling and testing, other 
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program considerations such as handicapped, hearings, private school 
participations authority and responsibility, and finance management. 
Other chapters of the manual discuss the purpose of Title III, details 
of local program development and operation, funding and fiscal considerations, 
and reporting requirements for State educational agencies. An annual 
report is presented covering areas and scope of activities to report, 
dissemination activities, instructions for completing the annual report 
and financial information to include. Several appendices contain formats 
for preparing plans, certification forms, copyright guidelines, grant 
terms and conditions, a bibliography, and UCLA's Center for the Study of 
Evaluation hierarchical objectives charts. 
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Basic Program Plan, Accountable Learning Systems 
Durham, North Carolina 27701 
National Laboratory for Higher Education 
April 1, 1972 

The basic components and evaluation strategies of the Accountable Learning 
Systems (ALS) program, which is designed to increase instructional effect- 
iveness and relevance in two year colleges, are described in this volume. 
The overall objective of the program is to develop tested products for 
training faculty and administrators in clarifying college goals and ob- 
jectives so that individualized courses of instruction may be systematic- 
ally developed. The program description discusses: 1) significance of 
the problem to the target audience; 2) expected outcomes; 3) assumptions; 
4) overall strategy; and 5) program implementation. Numerous product 
descriptions and schedules are described for the following program com- 
ponents: 1) training for instructional accountability; 2) management 
support for instructional accountability; 3) curriculum development for 
instructional accountability; and 4) technical assistance and service. 
Extensive evaluation charts, which use a criterion-referenced approach, 
are included for determining the efficiency and effectiveness of the ALS 
program. These illustrate several evaluation strategies and modes of 
data collection and reflect the current status of evaluation plans and 
results. The charts contain the following information, when available: 

1) identification of the program aspect or product being evaluated; 

2) questions the evaluation is designed to answer; 3) performance criteria; 
4) the design, data collection, and analysis methods utilized; and 5) 

test results or the schedule dates for testing. 

Program Evaluation , 

Categorically Aided Programs Evaluation, Continual Evaluation of Proposals 

and Projects 
The University of the State of New York 
Albany, New York 12224 
The State Education Department 

The evaluation of the State of New York's categorically aided programs 
at several time points, from proposal submission to product evaluation, 
^ is briefly described in this booklet. Utilizing the CIPP model, which 
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was first developed by Stuff! ebeam, the objectives for the following 
evaluation components are listed; context, input, process, and product. 
The proposal elements which are examined for evaluation purposes are 
listed and the qualities and characteristics of worthwhile proposals are 
enumerated. Techniques for monitoring an on-going project are detailed 
and the characteristics of a worthwhile on-going project are listed. 
The two major techniques for final program evaluation are cited and the 
qualities of an exemplary project which merits dessimi nation are detailed. 
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Closing the Gap, a Report of the First Two Years of Experience with 

ESEA, Title 1 in New York State 
The University of the State of New York 
Albany, New York 12224 
The State Education Department 
August, 1968 

The major program results of the State of New York's first two years 
experience with the Elementary and Education Act, Title I funding are 
summarized in this booklet. This funding was provided in 1965-67 for 
special educational programs designed to broaden and strengthen education 
for deprived students. Tabular results for the two year program for 
children participating in the following programs and their level of 
improvement are provided: reading programs; pupil personnel services; 
English language arts; and mathematics programs. Selected results for 
the first year are provided on the following programs: grade 4 gains in 
word knowledge and discrimination, reading, and arithmetic concepts; 
and remedial reading for low achievers in high school. Program statis- 
tics and selected results for the second funded year are provided on: 
1) attitude changes in grades 1-8; 2) types of programs funded; 3) methods 
used for achieving program goals; 4) type of measuring devices used; 
5) reading programs by grade level; 6) types of evaluation used; and 
7) effectiveness of reading programs. Four specific programs are de- 
scribed: 1) a corrective reading program with a cultural enrichment 
program; 2) preparing high school students for college; 3) a pupil transfer 
program; and 4) a comprehensive reading program for grades 1-12. Con- 
clusions and implications derived from the program are then summarized. 

Research and Evaluation 

Experimental Assessment of an Incentive Program to Enhance School Learning: 

A Pilot Study, Final Report to Franklin-McKinley School District 
Jung, Steven, M., Lipe, Dewey, Canter, Shelly 
P. 0. Box 1113, Palo Alto, California 94302 
American Institutes for Research 
July 15, 1971 

A pilot study to ascertain the effects of utilizing six educational 
incentive models for rewarding teachers, students, and parents in producing 
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student learning gains is reported. The purposes of the pilot study were 
to: determine if a successful experimental field study of incentives 
could be Implemented; develop and refine the methodology necessary for 
a large-scale field study; and ascertain preliminary estimates of incentive 
effects for assessing the potentiality of further research on the topic. 
The study specifically examined the effects of incentives in acquiring 
mathematics and reading skills for children, grades 1-3, over a 4-5 week 
period. Three experimental schools, two active control schools, and one 
passive control school were involved in the project. The instructional 
material was individualized for each student and incentives were pro- 
vided if the class as a whole achieved a required standard of mastery. 
The various incentive models, types of incentives, and their effects on 
student performance and student, parent, and teacher attitudes are 
discussed. Two techniques for monitoring the instructional process are 
described. Tabular data results are presented on administered pre-, 
and post-, standardized tests, and on criterion-referenced tests. 
Appendixed are a comprehensive list of objectives for primary reading 
and mathematics and various letters, questionnaires, procedures, etc. 
used for the pilot study. 

Statistics . 

How to Lie With Statistics 
Huff, Darren 

W. W. Norton & Company, New York 
1954 

This basically non-mathematical book introduces the layman to common 
statistical measurements and techniques by repeatedly exemplifying their 
use to mislead, confuse, inflate, sensationalize, and oversimplify. 
A distinction is made between mode, median and arithmetic means; the 
term "average" is dismissed as meaningless when used without qualification. 
Stratified, random and representative samples are distinguished and the 
selection of biased samples is explored. Consideration is given to 
misleading statistics about distributions when it is not specified 
whether the distribution is normal or skewed. Ranges, degree of signif- 
icance, degree of precision, probable error and standard error are defined; 
then certain data are shown to be of little value without the presence 
of one or more of these measurements. Various deceptions in line, bar, 
and pictorial graphs such as distortion of measurement, divisions, 
showing only part of the total graph, and changing the coordinates of the 
ordinate and abscissa are exemplified. Studies are reported of biases 
which result from improper questionnaire and interview survey techniques. 
The use of deceptive words, linear regression or cyclical variations, 
and cause-effect are also demonstrated. In conclusion, five simple 
questions are formulated which should be posed and answered to avoid 
being misled by statistics. 
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